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in  importance.  Beginning  physics  students,  however,  judged  the  definitions  as 
more  important.  These  results  suggest  that  sentence  form  is  a  salient  text 
feature  for  beginning-level  students.  Beginners  lack  the  knowledge  necessary 
to  judge  the  importance  of  content  directly,  but  they  have  developed  general 
rules  about  what  types  of  information  are  often  important  in  physics.  By  con¬ 
trast,  sentence  form  is  not  relevant  for  people  with  no  training  in  physics 
--  they  have  no  expectations  at  all  regarding  what  types  of  information  should 
be  important;  and  experts  pay  little  attention  to  sentence  form  because  they 
have  rich  content  schemas  which  enable  them  to  judge  importance  directly. 

These  results  have  theoretical  implications  for  understanding  content  schema 
development  and  also  have  practical  implications  for  the  writing  of  textbooks. 
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Novice  Rules  for  Assessing  Importance  in  Scientific  Texts: 

Executive  Summary 

In  learning  from  text,  finding  the  important  information  is  crucial,  particularly  for  a  novice 
reader  (i.e. .  one  who  is  unfamiliar  with  the  content  domain).  Such  information  serves  as  a 
focus  around  which  to  organize  related  details.  This  research  examined  novice  strategies  for 
determining  what  is  important  in  texts  containing  unfamiliar  content.  The  research  reported 
here  provides  the  theoretical  background  for  related  research  which  demonstrated  that 
novices’  inability  to  correctly  identify  important  text  information  can  influence  their  learning  in 
adverse  ways. 

This  study  specifically  examined  how  novices  process  definitions  and  facts,  two  types  of 
information  that  are  both  common  and  important  in  formal  domains.  The  results  Indicate 
that  in  judging  importance,  novice  readers  tend  to  focus  on  definitions  to  the  exclusion  of 
facts.  This  study  had  novices  judge  importance  in  passages  which  contained  definitions  and 
facts  that  were  (through  experimental  manipulation)  identical  to  each  other  in  content.  Even 
though  the  definitions  and  facts  contained  the  same  information,  novices  judged  the 
definitions  to  be  more  important.  This  was  not  true  of  experts,  who  judged  definitions  and 
facts  containing  the  same  content  to  be  equal  in  importance. 

These  findings  suggest  how  novices  begin  to  develop  a  "content  schema"  for  expository 
knowledge  domains.  Novices  learn  that  certain  easily  identifiable  types  of  information  (e  g., 
definitions)  are  more  important--a  reasonable  general  rule.  However,  they  judge  importance 
on  the  basis  of  information-type  category  without  regard  for  content,  considering  the  same 
information  to  be  more  important  if  it  is  expressed  in  the  form  of  a  definition.  Thus  novice 
readers  can  be  very  sensitive  to  the  form  in  which  information  is  presented,  in  that  minor 
variations  in  wording  (i.e.,  saying  something  "is  defined  as")  can  influence  novices' 
perception  of  how  important  that  information  is.  Accordingly,  writers  can  unintentionally 
signal  particular  Information  as  important  through  wording  selections  of  this  kind  if  they  are 
unaware  of  novice  preconceptions  about  the  types  of  information  that  are  important.  On  the 
other  hand,  knowledge  of  novice  rules  can  be  used  in  conjunction  with  rhetorical  indicators 
of  importance  (i.e.,  underlining,  signaling,  structural  changes,  etc.)  to  emphasize  important 
content  and  de-emphasize  less  critical  information.  Additionally,  knowledge  of  novice  content 
schemas  and  how  they  differ  from  those  of  experts  gives  us  a  basis  for  designing  instruction 
in  a  way  that  helps  novices  develop  better  text-learning  skills. 
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Abstract 

This  study  complements  research  indicating  that  content  area  novices  judge  importance 
in  texts  on  the  basis  of  sentence  type  (e  g.,  whether  sentences  are  definitions  or  facts). 
Subjects  varying  in  expertise  judged  the  importance  of  sentences  in  physics  texts  which 
were  presented  as  definitions  or  facts.  The  definitions  and  facts  were  identical  in 
substantive  content.  Experts  and  subjects  without  physics  training  judged  these  variants  as 
equal  in  importance.  However,  beginning  physics  students  judged  definitions  as  more 
important.  These  results  suggest  that  sentence  form  Is  a  salient  text  feature  for  beginning- 
level  students.  They  lack  the  knowledge  necessary  to  judge  the  importance  of  content 
directly,  but  have  developed  general  rules  about  what  types  of  information  are  often 
important  In  physics.  Sentence  form  is  not  relevant  for  people  without  physics  training-thay 
do  not  have  expectations  regarding  what  types  of  Information  are  important  Sentence  form 
also  has  little  influence  on  experts,  who  have  rich  content  schemas  allowing  them  to  judge 
importance  directly.  These  results  have  theoretical  implications  for  understanding  content 
schema  development,  and  practical  implications  tor  textbook  writers. 
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Novice  Rules  for  Assessing  Importance  in  Scientific  Texts 

Diana  Dee-Lucas  &  Jill  H.  Larkin 
Carnegie-Mellon  University 

Scientific  textbooks  are  typically  densely  packed  with  complex  information,  including 
equations,  symbols,  and  specialized  terms.  Consequently,  it  can  be  very  difficult  for 
students  who  are  unfamiliar  with  scientific  subject  matter  to  distinguish  the  important  content 
from  the  elaborative  information  when  reading  this  type  of  text.  The  purpose  of  the  present 
research  was  to  investigate  rules  used  by  novice  readers  (i.e. ,  readers  who  are  unfamiliar 
with  the  text  content  domain)  In  determining  what  is  important  in  scientific  texts. 

There  are  various  sources  of  information  that  novices  readers  could  use  in  assessing 
Imponance.  Most  research  has  focused  on  how  text-based  indicators  of  importance  such  as 
text  structure  and  signaling  devices  (eg.,  underlining,  adjunct  questions,  staging, 
typographical  cuing,  etc.)  influence  novice  readers'  attention.  These  textual  manipulations 
are  'content-free,”  in  that  their  effects  should  not  depend  on  the  nature  of  the  text  content 
or  the  expertise  of  the  reader  in  contrast,  a  'content-specific"  source  of  information  for 
assessing  Importance  in  texts  is  the  reader's  'content  schema"  (Kieras,  1985).  A  content 
schema  consists  of  domain-specific  knowledge  about  how  Information  in  a  content  area  is 
typically  organized,  including  what  is  important 

Past  research  has  largely  ignored  the  role  of  content-based  schemas  in  the 
comprehension  of  novice  readers  This  is  because  it  has  typically  been  assumed  that  novice 
readers  who  are  unfamiliar  with  the  content  domain  of  a  text  would  lack  such  a  schema 
(see  Kieras,  1985,  for  discussion).  However,  recent  work  by  Dee-Lucas  and  Larkin  (1986) 
suggests  that  novices  do  develop  a  rudimentary  "content  schema"  for  scientific  content 
domains.  This  content  schema  consists  of  rules  specifying  the  types  of  information  (i.e  , 
definitions,  facts,  etc.)  that  are  important  to  that  domain  In  their  research,  Dee-Lucas  and 
Larkin  compared  the  importance  judgements  of  expert  and  novice  physicists  for  different 
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types  of  information  in  physics  texts.  They  found  that  both  groups  judged  definitions  to  be 
more  important  than  facts.  However,  the  novices  were  even  more  likely  than  the  experts  to 
judge  definitions  as  important  and  facts  as  unimportant,  suggesting  that  they  had 
*  overgeneralized"  this  rule.  The  experts  discriminated  between  important  and  unimportant 
information  within  the  type  categories  to  a  greater  degree  than  the  novices,  indicating  a 
more  articulated  content  schema.  Thus  novices  are  Inaccurately  identifying  the  important 
text  content  on  the  basis  of  these  overgeneralized  rules. 

This  research  comparing  the  judged  importance  of  definitions  and  facts  did  not, 
however,  control  for  content  differences  between  the  two  sentence  types  Thus  novices  in 
these  studies  may  have  been  basing  their  importance  judgements  on  some  feature  of  the 
content  that  differed  between  definitions  and  facts,  rather  than  on  the  definition/fact 
distinction  per  $e.  For  example,  novices  may  have  considered  definitions  to  be  more 
important  than  facts  because  the  definitions  may  have  contained  more  new  terms,  if  this 
were  the  case,  then  the  rule  used  by  the  novices  would  be  that  statements  with  new  terms 
are  more  Important  than  statements  involving  known  terms,  rather  than  a  rule  that  definitions 
are  more  important  than  facts. 

The  current  study  provides  a  strong  test  of  the  hypothesis  that  novices  are  using  a 
rule  involving  a  pure  sentence  type  distinction;  in  other  words,  that  novices  consider 
definitions  to  be  important  simply  because  they  are  definitions,  regardless  of  their  content. 
This  was  tested  by  comparing  novices'  and  experts’  relative  sensitivity  to  the  form  (i.e. , 
definition  or  fact)  In  which  information  is  presented  in  physics  passages.  Experts  and 
novices  judged  the  importance  of  target  sentences  in  physics  texts  when  they  were 
presented  as  definitions  or  facts.  These  target  sentences  contained  information  which  could 
be  stated  as  facts  or  definitions  through  minor  variations  in  wording  which  did  not  alter  the 
primary  sentence  content.  In  this  way,  the  content  of  the  definitions  and  facts  remained 
constant  and  only  the  form  in  which  the  information  was  presented  varied.  Thus  unlike 
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earlier  research,  this  study  controlled  for  possible  sentence-type  effects  due  to  content 
differences  between  definitions  and  facts.  Two  versions  of  two  physics  passages  were  used, 
each  version  being  identical  except  for  the  form  in  which  the  target  sentences  were 

presented.  The  importance  Judgements  of  the  novices  and  experts  for  the  fact  and 

definition  versions  of  the  target  sentences  were  compared  to  see  to  what  extent  the 

sentence  form  influenced  the  perceived  importance  of  the  information. 

Method 

Stimulus  Materials 

One  passage  was  about  work  and  energy  and  one  dealt  with  fluid  statics.  Each  was 
about  50  sentences  long.  One  contained  9  target  sentences  and  one  had  1 1  target 

sentences. 

The  definition  and  fact  versions  of  the  target  sentences  differed  in  that  definitions 

always  Included  'is  defined  as,'  and  thus  were  signaled  as  being  definitions.  In  the  fact 
versions,  'is  defined  as"  was  dropped  or  replaced  with  'is  represented  as."  "is  calculated 

as,"  or  'is  Indicated  by."  Thus  the  facts  were  "non-definitions"  in  that  in  place  of 

definition  signaling  they  contained  phrases  indicating  that  the  sentence  was  presenting 

attributive  information  about  the  sentence  topic  (as  opposed  to  criteria!  attributes  defining  the 
sentence  topic).  Examples  of  the  definition  and  fact  versions  of  some  of  the  target 
sentences  are  shown  in  Table  1.  There  were  two  versions  of  each  passage.  In  one 
version,  the  odd-numbered  target  sentences  were  definitions  and  the  even-numbered  were 

facts;  in  the  second  version  this  was  reversed 

I  risen  Table  1  about  here 

Each  of  the  target  sentences  was  classified  according  to  its  level  in  the  hierarchical 
structure  of  the  passage.  The  procedure  used  for  the  structural  analysis  is  reported  in  Dee- 
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Lucas  and  Larkin  (1986).  This  analysis  produced  a  hierarchy  with  the  main  topics  or 
concepts  occurring  at  the  highest  levels  and  modifying  information  occurring  at  the  lower 
levels.  Modifying  information  consisted  of  examples,  attributes  and  properties,  derivations 
(l.e.,  Information  Implied  by  or  derived  from  higher  level  information),  explanations,  sub-topics, 
and  preconditions  (l.e.,  necessary  conditions  for  a  rule,  principle,  or  fact  to  hold  true).  The 
hierarchical  analysis  was  performed  at  the  sentence  level.  There  were  7  sentences  at  level 
1  (the  most  superordinate  level),  6  sentences  at  level  2,  and  7  sentences  at  level  3. 
Hierarchical  level  was  Included  as  a  variable  in  the  data  analyses  to  see  whether  perceived 
Importance  was  influenced  by  level,  and  If  this  variable  Interacted  with  sentence  form  (i.e., 
definition  or  fact). 


Subjects 


The  novices  were  24  undergraduates  with  2  or  3  semesters  of  college  physics. 
Novices  with  this  level  of  physics  training  were  selected  to  insure  that  the  novice  group  had 
had  enough  exposure  to  physics  to  have  developed  information-type  rules,  but  had  not 
approached  the  expert  level  in  training.  The  24  experts  had  completed  at  least  one  year  of 
graduate  study  in  physics. 

Two  control  groups  were  also  run  in  the  experiment  to  see  if  expert-novice  differences 
in  the  perceived  importance  of  the  target  sentences  were  due  to  differences  in  educational 
level  (i.e.,  undergraduate  vs.  graduate  level  training)  as  opposed  to  differences  in  physics 
knowledge.  The  two  control  groups  were  selected  so  as  to  differ  in  their  educational  level 
in  the  same  manner  as  the  two  experimental  groups.  However,  none  of  the  control  group 
subjects  had  taken  any  college-level  physics,  so  they  were  similar  to  each  other  in  terms  of 
their  physics  knowledge.  The  undergraduate  control  group  consisted  of  24  undergraduates; 
the  graduate  student  control  group  consisted  of  people  who  had  completed  at  least  i  year 
of  graduate  training  in  the  humanities  or  social  sciences  Although  this  group  will  be 
referred  to  as  the  graduate  student  control,  it  included  some  post-doctoral  researchers  and 
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faculty.  This  was  also  true  of  the  corresponding  expert  experimental  group. 

Although  the  control  groups  were  specifically  selected  to  control  for  educational 
differences,  they  would  also  indicate  differences  in  perceived  importance  due  to  age, 
maturity,  and  verbal  ability.  In  the  case  of  verbal  ability,  it  is  reasonable  to  assume  that 
graduate  students  in  the  social  sciences/humanities  would  be  as  high  in  verbal  ability  as 
graduate  students  in  physics.  Similarly,  It  is  likely  that  undergraduates  attending  the  same 
university  are  roughly  equivalent  in  verbal  ability. 

Procedure 

The  subjects  were  given  one  version  of  each  passage.  They  were  told  to  read  each 
passage  carefully,  then  rate  the  importance  of  each  sentence  on  a  scale  from  1  (most 
Important)  to  5  (least  important),  and  then  indicate  the  10  most  important  sentences  in  each 
passage.  The  instructions  for  the  rating  task  indicated  that  each  rating  should  be  used  at 
least  once.  All  of  the  sentences  were  rated,  but  only  the  ratings  for  the  target  sentences 
were  analyzed.  The  order  in  which  the  passages  were  read  and  the  versions  of  the 
passages  received  were  counterbalanced. 

The  novice  and  the  undergraduate  control  groups  were  told  that  in  completing  the 
tasks,  they  were  to  indicate  which  sentences  they  thought  would  be  most  important  to  learn 
if  they  were  going  to  be  tested  on  the  passage  content.  The  expert  and  the  graduate 
student  control  groups  were  told  to  pretend  that  they  were  teaching  a  course  and  indicate 
which  sentences  they  thought  were  most  important  for  their  students  to  learn.  These 
instructions  match  those  used  by  Dee-Lucas  and  Larkin  (1986)  in  their  initial  research  on 
expert-novice  differences  in  perceived  importance.  The  instructions  were  designed  to 
compare  what  novices  think  they  should  learn  with  what  experts  (their  instructors)  think 


novices  should  learn. 
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Results 

The  data  from  the  two  dependent  measures  were  analyzed  in  two  ways.  The  ratings 
data  were  analyzed  using  a  multiway  frequency  analysis.  This  analysis  fits  a  loglinear  model 
to  categorical  data.  The  number  of  responses  in  each  rating  category  (1  through  5)  for 
each  sentence  type  (definition  and  fact)  occurring  at  each  level  (1  through  3)  was  tabulated 
for  each  of  the  subject  groups.  The  multiway  frequency  analysis  was  performed  on  the  total 
number  of  responses  occurring  in  each  of  these  cells. 

The  data  from  the  sentence  selection  task  (i.e. ,  select  the  10  most  important 
sentences)  were  analyzed  with  a  logistic  regression  The  variables  entered  in  the  analysis 
for  each  target  sentence  for  each  subject  were  sentence  type  (definition  or  fact),  level  (1 
through  3),  and  subject  group  (novice  or  expert).  The  dependent  measure  was  whether  or 
not  the  sentence  had  been  selected  as  one  of  the  most  important. 

The  data  from  the  two  control  groups  were  submitted  to  identical  analyses.  The 
results  of  these  analyses  were  compared  to  the  results  of  the  corresponding  analyses  of  the 
experimental  group  data  to  determine  if  expert-novice  differences  were  also  reflected  in 
differences  between  graduate  students  and  undergraduates  who  had  had  no  advanced 
physics  training. 

Ratings  Data 

Experimental  groups.  The  multiway  frequency  analysis  of  the  ratings  data  indicated 
that  the  best-fitting  model  was  a  hierarchical  model  including  the  type  x  group  interaction 
and  the  main  effect  of  level  (^2  =  23.80,  df=  30,  p<  .79).  The  mean  ratings  predicted  by  the 
model  for  the  type  x  group  interaction  are  shown  in  Figure  la.  The  predicted  means  for 
the  novices  are  1.67  when  the  sentence  was  in  the  form  of  a  definition  and  1.89  when  it 
was  in  the  form  of  a  fact.  For  experts,  the  predicted  ratings  are  1.79  for  definitions  and 
1.82  for  facts.  This  interaction  Indicates  that  novices  were  influenced  in  their  importance 
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ratings  by  the  form  in  which  the  information  was  presented.  They  considered  the  same 
content  to  be  more  important  if  it  was  stated  as  a  definition  as  opposed  to  a  fact.  The 
experts,  on  the  other  hand,  did  not  appear  to  base  their  ratings  on  sentence  form;  there  is 
very  little  difference  in  their  predicted  mean  ratings  for  definition  and  fact  versions  of  the 
target  sentences. 

Insert  Figure  1  about  here 

The  parameter  estimates  for  the  main  effects  and  interaction  for  the  complete  model 
are  shown  In  Table  2.  Because  of  the  usual  constraints  placed  on  the  model,  ail  parameter 
estimates  for  each  main  effect  and  interaction  are  constrained  to  sum  to  zero.  Therefore, 
for  all  effects  the  magnitude  of  the  parameter  estimates  for  each  variable  are  the  same  but 
In  the  opposite  direction.  The  ratios  of  the  estimates  to  the  standard  errors  indicate  the 
degree  to  which  the  parameter  estimates  differ  from  zero. 

insert  Table  2  about  here 

The  parameter  estimates  for  the  main  effect  of  group  show  little  difference  between 
the  experts  and  novices  in  their  use  of  the  five  rating  categories.  The  largest  differences 
occurred  in  the  use  of  rating  5  (the  lowest  rating)  and  rating  3  (the  middle  rating).  The 
novices  tended  to  use  the  rating  3  category  more  often  than  the  experts  (as  indicated  by 
the  positive  parameter  estimate),  while  the  experts  tended  to  use  the  lowest  rating  more 
often  than  the  novices.  This  suggests  that  the  experts  rated  the  target  sentences  lower  in 
importance  than  the  novices. 

The  parameter  estimates  for  the  main  effect  of  type  indicate  that  the  largest  difference 
occurred  in  the  rating  3  category  (the  middle  rating).  The  negative  parameter  estimate 
Indicates  that  this  rating  was  used  more  often  with  facts  than  definitions.  There  were  also 
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smaller  differences  in  the  use  of  the  first  two  rating  categories,  with  the  definitions  rated  1 
or  2  more  often  relative  to  facts.  This  indicates  that  definitions  were  rated  higher  overall  in 
Importance  than  facts. 

The  type  x  group  interaction  estimates  indicate  that  the  greatest  differences  between 
experts  and  novices  in  rating  definitions  and  facts  occurred  in  the  first  two  rating  categories. 
The  novices  were  more  likely  than  experts  to  give  a  target  sentence  a  rating  of  1  if  it  was 
In  the  form  of  a  definition,  and  somewhat  more  likely  to  rate  it  a  2  if  it  was  in  the  form  of 
a  fact.  The  opposite  was  true  for  the  experts,  relative  to  the  novices. 

Level  had  a  very  strong  influence  on  the  target  sentence  ratings,  as  shown  by  the 
large  parameter  estimates  for  this  effect.  Level  t  target  sentences  tended  to  be  rated  as 
most  Important,  indicated  by  the  large  positive  estimate  for  the  rating  i  category.  The 
ratings  for  level  2  target  sentences  were  spread  over  the  categories  without  any  strong 
clustering  in  any  one  category;  none  of  the  parameter  estimates  for  level  2  sentences 
differed  from  zero  by  more  than  two  standard  errors.  Level  3  target  sentences  tended  to 
be  rated  as  3  or  4  in  importance,  indicated  by  the  positive  parameters  for  these  ratings 
categories.  Level  did  not  interact  with  type  or  group  in  influencing  the  ratings  (i.e  . 
Including  these  effects  reduced  the  fit  of  the  model). 

Control  groups.  The  multiway  frequency  analysis  of  the  control  group  data  indicated 
that  the  best-fitting  model  included  only  the  main  effects  of  group  and  level  (A  ?  =  28.64, 
df-40,  p<  91).  The  Inclusion  of  the  type  x  group  interaction  or  the  main  effect  of  type 
reduced  the  fit  of  the  model  to  the  data  set.  The  mean  ratings  predicted  by  the  model 
with  the  type  x  group  interaction  included  are  presented  in  Figure  1b  for  comparison  with 
the  corresponding  experimental  group  means.  As  Figure  fb  shows,  there  was  no  difference 
between  the  undergraduate  and  graduate  controls  in  the  influence  of  sentence  type  on  the 
mean  ratings  of  the  target  sentences  Additionally,  the  lack  of  a  main  effect  of  sentence 
type  indicates  that  the  form  in  which  the  target  sentences  were  presented  did  not  influence 
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the  control  group  ratings  (i.e.,  they  did  not  consider  definitions  to  be  important  than  facts). 

The  parameter  estimates  for  the  loglinear  model  including  the  main  effects  of  group 
and  level  are  shown  in  Table  3.  The  estimates  for  the  main  effect  of  group  indicate  that 
the  undergraduates  and  graduates  differed  primarily  in  their  use  of  ratings  1  and  4.  The 
positive  estimate  for  the  undergraduates  for  rating  category  1  indicates  that  they  gave  the 
target  sentences  a  rating  of  1  more  often  than  the  graduate  controls.  The  opposite  was 
true  for  the  rating  4  category,  with  the  graduates  using  this  rating  more  often  relative  to  the 
undergraduates.  This  indicates  that  undergraduates  rated  the  sentences  as  more  important 
overall  than  the  graduate  controls.  This  effect  is  also  apparent  in  Figure  1b. 

insert  Table  3  about  here 

The  pattern  of  parameter  estimates  for  the  main  effect  of  level  is  similar  to  that 
obtained  with  the  experimental  groups.  Level  1  target  sentences  were  most  likely  to  receive 
a  rating  of  1,  indicated  by  the  large  positive  parameter  estimate  for  that  rating.  The  ratings 
given  to  level  2  target  sentences  were  spread  over  the  categories,  with  the  strongest 

clustering  In  the  rating  1  category  (though  this  parameter  estimate  was  much  smaller  than 
the  rating  1  estimate  for  level  1  sentences).  Level  3  target  sentences  were  most  likely  to 
receive  a  rating  of  4,  with  3  as  the  next  most  frequent  rating  category  for  this  level. 

Summary.  The  results  of  the  multiway  frequency  analyses  indicate  that  novices  base 

their  judgements  of  the  importance  of  text  information  on  the  form  in  which  the  information 
is  presented.  They  specifically  rate  the  same  content  as  more  important  when  it  is 
presented  as  a  definition  as  opposed  to  a  fact,  as  is  shown  in  Figure  la.  Expert  physicists 
are  not  Influenced  by  sentence  form  in  rating  importance,  presumably  basing  their 

importance  judgements  on  the  nature  of  the  sentence  content.  Similarly,  subjects  without 
advanced  physics  training  are  not  influenced  by  sentence  form,  so  that  they  appear  to 
behave  like  experts  in  rating  importance.  This  can  be  seen  in  Figure  1b.  This  is  most 
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likely  because  these  subjects  have  no  strong  expectations  about  the  relative  importance  of 
definitions  and  facts  in  physics  texts,  and  thus  are  not  influenced  by  this  text  feature.  The 
lack  of  a  type  x  group  interaction  In  the  control  group  data  indicate  that  expert-novice 
differences  in  the  perceived  importance  of  definitions  and  facts  are  not  due  to  differences  in 
educational  level. 

Sentence  Selection  Data 

Experimental  groups.  The  sentence  selection  data  were  analyzed  using  a  logistic 
regression.  The  regression  analysis  Indicated  that  a  good  fit  to  the  experimental  group  data 
was  provided  by  a  hierarchical  model  including  the  group  x  type  interaction  and  the  main 
effect  of  level  (A2- 8.69,  df=  7,  p<.276).  This  is  the  same  model  found  to  provide  the  best 
fit  for  the  ratings  data  from  the  experimental  groups.  The  predicted  mean  proportion  of 
target  sentences  selected  for  the  type  x  group  interaction  are  shown  in  Figure  2a  plotted  on 
a  logit  scale.  This  Interaction  is  very  similar  to  the  type  x  group  interaction  obtained  with 
the  ratings  data.  It  shows  that  the  novices  were  more  likely  to  select  a  target  sentence  as 
Important  when  it  was  presented  in  the  form  of  a  definition  than  a  fact,  while  the  experts 
were  relatively  unaffected  by  sentence  form  In  their  selection  of  the  important  sentences. 


Insert  Figure  2  about  here 


The  parameter  estimates  for  the  complete  logistic  regression  model  are  shown  in  Table 
4.  Unlike  the  estimates  for  the  multiway  frequency  analysis,  the  logistic  regression  estimates 
show  the  size  of  the  difference  between  the  means  for  the  two  variables  in  an  effect 
Therefore  only  one  parameter  estimate  Is  presented  for  each  main  effect  (two  for  the 
interaction)  and  the  ratios  indicate  the  size  of  the  difference  between  the  two  variables  in 


each  effect  and  Interaction. 
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Insert  Table  4  about  here 


The  negative  estimate  for  the  main  effect  of  type  indicates  that  subjects  tended  to 
select  more  target  sentences  when  they  were  presented  in  the  form  of  a  definition.  This  is 
consistent  with  the  main  effect  of  type  found  with  the  ratings  data  from  the  experimental 
groups.  The  estimate  for  the  main  effect  of  group  indicates  little  difference  between  the 
experts  and  novices  in  the  number  of  target  sentences  selected  as  important.  The  negative 
estimate  indicates  that  the  novices  selected  more  target  sentences  than  the  experts. 
However,  the  parameter  estimates  for  the  two  groups  differed  by  less  than  two  standard 
errors. 

The  type  x  group  interaction  estimates  indicate  that  novices  were  more  likely  to  select 
target  sentences  as  Important  when  they  were  in  the  form  of  a  definition  rather  than  a  fact. 
Relative  to  novices,  the  experts  were  more  likely  to  select  the  sentences  when  they  were  in 
the  form  of  a  fact.  These  findings  are  shown  by  the  positive  estimate  for  definitions  and 
the  negative  estimate  for  facts. 

The  very  large  parameter  estimates  for  the  main  effect  of  level  indicates  that  this 
variable  had  a  strong  effect  on  which  target  sentences  were  selected  as  important.  The 
negative  estimate  indicates  that  target  sentences  from  the  upper  levels  of  the  passage 
hierarchy  (level  1)  were  selected  more  often  than  the  target  sentences  from  the  lower  levels 
(level  3).  This  finding  is  also  consistent  with  the  strong  levels  effect  found  in  the  ratings 
data.  As  with  the  ratings  data,  there  was  no  indication  that  level  interacted  with  sentence 
type  or  group  in  influencing  sentence  selection. 

Control  groups.  The  logistic  regression  for  the  control  group  data  indicated  that  the 
best-fitting  model  was  a  hierarchical  model  including  the  main  effect  of  group  and  the  type 
x  level  Interaction  (A(2=1.78,  df=  5,  p<.879).  The  inclusion  of  the  type  x  group  interaction 
reduced  the  fit  of  the  model  to  the  data.  Thus  there  was  no  evidence  that  the  importance 
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judgements  of  the  undergraduate  and  graduate  control  groups  differed  in  the  degree  to 

which  they  were  influenced  by  sentence  type.  This  can  be  seen  in  Figure  2b,  which  shows 
the  predicted  mean  proportion  of  definitions  and  facts  selected  by  the  two  control  groups 
plotted  on  a  logit  scale. 

The  parameter  estimates  for  the  regression  model  including  the  effects  of  group  and 
the  type  x  level  Interaction  are  presented  in  Table  5.  Unlike  the  experimental  groups,  level 
did  not  have  a  linear  effect  on  the  number  of  target  sentences  selected  by  the  control 

groups.  It  was  therefore  entered  into  the  analysis  as  a  categorical  (as  opposed  to  a  linear) 
variable,  and  separate  parameter  estimates  were  obtained  for  each  level.  The  parameter 

estimates  presented  for  the  main  effect  of  level  represent  the  size  of  the  difference  between 
levels  1  and  2,  and  levels  2  and  3.  For  the  type  x  level  interaction,  the  parameter 

estimates  Indicate  the  si2e  of  the  difference  between  definitions  and  facts  at  each  level. 


Insert  Table  5  about  here 


The  negative  parameter  estimate  for  the  main  effect  of  type  indicates  that  the  control 
groups  were  more  likely  to  select  a  target  sentence  when  it  was  in  the  form  of  a  definition 
than  a  fact.  However,  the  type  x  level  interaction  indicates  that  the  effect  of  type  varied 
with  level.  The  negative  parameter  estimate  for  level  1  indicates  that  facts  were  selected 
more  often  than  definitions  at  the  top  level,  while  the  positive  parameters  for  levels  2  and  3 
show  that  definitions  were  more  likely  to  be  selected  than  facts  at  the  lower  levels.  The 
predicted  cell  means  for  this  Interaction  are  presented  in  Figure  3.  Figure  3  shows  that  the 
main  effect  of  sentence  type  is  due  primarily  to  a  very  large  sentence  form  effect  at  level  3. 

Insert  Figure  3  about  here 


This  finding  suggests  that  subjects  without  physics  training  may  tend  to  judge  details  (low- 


Novice  Rules 


1  & 

level  information)  in  physics  texts  as  being  more  important  when  they  are  presented  as 

definitions  than  facts.  However,  this  sentence  type  difference  is  based  on  very  few  data 
points,  as  most  level  3  sentences  were  not  selected  as  important.  The  mean  number  of 
sentences  from  level  3  selected  by  the  control  groups  were  2.3  for  the  undergraduates  and 
1.6  for  the  graduates.  Additionally,  this  type  x  level  interaction  was  not  found  in  the  ratings 
data  for  the  control  groups.  Therefore,  it  is  possible  that  this  particular  effect  is  not 

replicable. 

The  negative  parameter  estimate  for  the  main  effect  of  group  indicates  that 
undergraduates  selected  more  target  sentences  as  important  than  the  graduate  control 
group.  This  Is  consistent  with  the  finding  in  the  ratings  data  that  undergraduates  tended  to 
rate  the  target  sentences  higher  in  importance  relative  to  the  graduate  controls. 

The  positive  parameter  estimates  for  the  main  effect  of  level  indicate  that  the  number 
of  sentences  selected  as  important  decreased  with  level.  The  size  of  the  estimates  show 
that  the  drop  in  the  number  of  target  sentences  selected  as  important  was  much  greater 

between  levels  2  and  3  than  between  levels  1  and  2.  This  can  also  be  seen  in  Figure  3. 

Summary.  The  results  of  the  sentence  selection  task  analyses  are  consistent  with  the 
findings  from  the  ratings  data.  Sentence  form  Influenced  the  importance  judgements  of 

novices,  with  novices  selecting  more  target  sentences  when  they  were  presented  In  the  form 
of  a  definition  (see  Figure  2a).  Sentence  form  had  very  little  effect  on  the  sentences 

selected  by  experts.  There  again  was  no  type  x  group  interaction  in  the  control  group 

data,  as  shown  in  Figure  2b,  indicating  that  expert-novice  differences  are  not  due  to 
differences  In  educational  level.  Sentence  form  did  have  some  influence  on  the  judged 

importance  of  sentences  for  the  control  groups.  The  regression  analysis  indicated  that  the 
control  groups  tended  to  select  more  sentences  from  level  3  when  they  were  in  the  form  of 
a  definition  than  a  fact.  Thus  it  Is  possible  that  there  Is  a  bias  towards  considering  low- 
level  definitions  as  Important  in  subjects  who  do  not  have  scientific  backgrounds 
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Discussion 

Although  much  research  has  examined  the  knowledge  representations  of  experts, 
relatively  little  is  known  about  the  knowledge  structures  of  novices.  This  is  most  likely 
because  novice  representations  are  assumed  to  be  more  variable  than  those  of  experts,  and 
thus  are  studied  less.  However,  how  novices  represent  unfamiliar  content  domains  and  the 
nature  of  the  changes  occurring  in  these  representations  has  important  educational 
Implications  for  facilitating  learning  from  text.  Characterizations  of  the  knowledge  structures 
used  by  novices  to  govern  text  processing  indicate  how  text  should  be  structured,  both 
globally  and  locally,  to  maximize  the  probability  that  novices  will  learn  the  important  text 
content. 

This  study  examined  one  feature  of  novice  knowledge  representations-rules  for 
assessing  Importance  in  unfamiliar  scientific  domains.  It  investigated  the  nature  of  novice 
scientists’  content  schema  for  physics  subject  matter,  specifically  testing  the  hypothesis  that 
novices  judge  importance  in  physics  texts  on  the  basis  of  sentence  form.  This  was  done  by 
using  minor  wording  changes  to  transform  sentences  into  definitions  or  facts,  and  looking  at 
how  this  altered  their  perceived  importance.  Novices  judged  the  same  information  as  more 
important  when  it  was  presented  as  a  definition,  indicating  that  they  based  their  judgements 
on  sentence  form.  The  form  of  the  Information  did  not  affect  the  importance  judgements  of 
experts.  The  experts  presumably  judged  the  importance  of  the  information  according  to  its 
content,  rather  than  its  form.  This  type  by  group  interaction  was  not  found  with 
undergraduate  and  graduate  controls,  so  this  interaction  is  not  due  to  differences  in  the 
educational  level  of  the  physics  experts  and  novices. 

These  findings  indicate  that  novices  who  have  had  some  college  physics  develop  a 
rule  that  definitions  are  more  important  than  facts.  This  rule  is  not  inappropriate-experts 
also  consider  definitions  to  be  more  important  than  facts  in  physics  passages  in  which 
content  differences  between  the  two  sentence  types  have  not  been  controlled  (Dee-Lucas  & 
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Larkin,  1986).  These  expert  data  suggest  that  definitions  in  physics  textbooks  may  contain 
information  that  is  particularly  important  for  understanding  these  texts.  However,  the  results 
of  the  current  study  Indicate  that  novices  judge  importance  on  the  basis  of  sentence  type 
category  without  regard  for  sentence  content.  They  judged  the  same  information  as  more 
important  when  it  was  presented  as  a  definition,  indicating  that  they  consider  definitions  to 
be  important  regardless  of  their  content.  This  suggests  that  novices  are  applying  this 
Information-type  rule  too  rigidly,  in  that  they  are  not  distinguishing  between  important  and 
unimportant  Information  within  the  definition  and  fact  categories.  In  this  sense,  they  have 
'overgeneralized"  the  rule.  This  overgeneralization  results  In  novices  systematically  mis- 
identifying  the  important  text  information  by  missing  important  facts  and  focusing  on 
unimportant  definitions. 

These  findings  parallel  those  on  expert-novice  categorization  of  problem  types  in 
physics  and  math  (Chi,  Feltovich,  &  Glaser,  1981;  Schoenfeld  &  Herrmann,  1982).  This 
research  on  problem  identification  has  found  that  novices  are  influenced  by  surface-level 
features  in  their  early  categorization  schemes,  but  move  to  a  more  expert-like  representation 
with  Instruction.  In  the  current  study,  the  novice  rules  are  also  based  on  superficial 
characteristics  (eg.,  category  type  membership).  However,  the  resulting  importance 
judgements  based  on  these  rules  are  not  radically  different  from  those  of  experts,  Indicating 
that  this  feature  is  related  to  some  degree  to  the  underlying  content  structure  defining 
Importance  within  the  domain  of  physics 

Information-type  rules  of  the  kind  demonstrated  here  can  be  thought  of  as  the 
foundation  for  a  content  schema.  A  content  schema  includes  knowledge  about  what 
Information  Is  Important  in  a  content  area.  The  results  of  the  current  study  suggest  that  an 
early  stage  in  the  development  of  a  content  schema  for  scientific  subject  matter  may  be  the 
specification  of  rules  indicating  what  types  of  information  are  important  in  that  content 
domain.  These  information-type  rules  have  important  learning  consequences.  Previous 
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research  has  found  that  novices  spend  more  time  on  definitions  than  facts  when  reading 
physics  passages,  recall  more  definitions  than  facts  afterwards,  and  include  more  definitions 
than  facts  In  their  summaries  of  physics  texts  (Dee-Lucas  &  Larkin,  1985,  1986).  Thus 
these  Information-type  rules  appear  to  influence  novice  readers’  attentional  processes  during 
reading,  as  well  as  the  macrostructure  they  develop  for  physics  texts. 

Unlike  novices,  the  importance  judgements  of  the  control  subjects  with  no  college 
physics  were  largely  unaffected  by  sentence  form.  These  subjects  therefore  appear  similar 
to  experts.  This  Is  most  likely  because  they  have  no  strong  expectations  about  what  types 
of  content  should  be  important  in  physics  texts.  Thus  the  control  subjects  were  not 
Influenced  by  sentence  form  because  they  lack  a  physics-relevant  content  schema;  the 
experts  were  not  Influenced  by  form  because  they  have  developed  a  much  more  refined 
schema  which  Indicates  the  importance  of  specific  information  within  the  domain  of  physics. 
The  experts'  content  schema  includes  a  finer-grained  analysis  of  the  importance  of  the 
Information  than  that  captured  by  sentence-type  classifications. 

This  suggests  an  inverted  U-shaped  relationship  among  naive,  novice,  and  expert 
physicists  in  their  sensitivity  to  sentence  form.  Both  naive  and  expert  subjects  are  not 
influenced  by  sentence  form;  they  judge  the  same  content  equal  in  importance  regardless  of 
its  form.  Novices,  on  the  other  hand,  are  'distracted’’  by  form  in  their  importance 
judgements,  in  that  they  alter  their  judgements  according  to  whether  the  information  is 
presented  as  a  definition  or  a  fact.  This  is  because  their  knowledge  of  physics  has  caused 
them  to  develop  a  general  expectation  that  definitions  are  more  important  than  facts  (an 
expectation  lacking  in  naive  subjects),  but  they  do  not  have  sufficient  knowledge  to  judge 
the  Importance  of  specific  content  within  these  categories  (as  experts  can),  and  thus  rely  on 
this  general  rule  based  on  sentence  form. 

A  similar  U-shaped  relation  has  been  found  among  beginning,  intermediate,  and  expert 
radiologists  in  diagnosing  x-ray  pictures.  Lesgold,  Feltovich,  Glaser.  &  Wang  (1981)  found 
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that  beginning  residents  and  experts  were  better  than  intermediate-level  radiologists  in  the 
diagnostic  reading  of  certain  classes  of  x-ray  films.  This  is  because  accurate  diagnosis 
Involves  an  interaction  between  the  physical  features  found  on  the  x-ray  and  the  radiologist's 
knowledge  of  the  relevant  contextual  features  which  constrain  the  possible  alternative 
diagnoses.  Beginning-level  radiologists  base  their  diagnoses  on  the  physical  features  of  the 
x-ray,  and  are  accurate  when  there  happens  to  be  a  match  between  their  interpretation  of 
those  features  and  the  actual  pathology;  expens  use  their  schematic  knowledge  to  interpret 
the  physical  features  in  the  context  of  other  relevant  Information  (such  as  the  patient's 
medical  condition),  and  thus  are  systematically  accurate  in  their  diagnoses.  Intermediate- 
level  radiologists  possess  some  schematic  knowledge,  but  this  knowledge  is  not  refined  or 
flexible  enough  to  provide  accurate  diagnoses--it  "distracts"  them  from  the  direct  physical 
features  used  by  beginners,  and  is  not  elaborate  enough  to  allow  them  to  pinpoint  the 
appropriate  alternative  in  the  same  manner  as  experts. 

Thus  both  with  physicists  and  radiologists,  there  is  a  stage  in  the  development  of 
expertise  in  which  novices  possess  a  primitive  schema  incorporating  very  general  information 
about  the  domain  (i.e. ,  what  types  of  information  are  typically  important  for  understanding 
physics;  what  contextual  features  are  relevant  for  interpreting  an  x-ray).  These  schemata  are 
not  sufficiently  refined  to  reliably  allow  accurate  performance,  and  actually  impair  novice 
performance  (relative  to  that  of  naive  subjects)  or  certain  tasks.  Similar  developmental 
trends  have  been  found  in  the  performance  of  children  on  a  variety  of  cognitive  tasks  (see 
Richards  &  Siegler,  1982).  These  findings  suggest  that  a  short-term  decrement  in  certain 
aspects  of  task  performance  may  be  a  necessary  consequence  of  the  early  forms  of 
schematic  knowledge  that  develop  in  the  course  of  acquiring  expertise  in  a  variety  of 
content  domains. 

This  study  was  a  strong  test  of  the  hypothesis  that  novices  are  influenced  by  sentence 
form  in  judging  importance,  in  that  all  variables  other  than  sentence  form  were  held  constant 
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In  the  target  sentences.  The  differences  in  the  perceived  importance  of  definitions  and  facts 
obtained  for  novices  were  small  in  absolute  terms.  This  is  to  be  expected  given  the  nature 
of  the  manipulation.  The  only  difference  between  the  definition  and  fact  versions  of  the 
target  sentences  was  in  the  use  of  the  words  'is  defined  as'-the  substantive  content  of  the 
sentences  and  their  location  in  the  texts  remained  the  same  for  both  sentence  versions 
Thus  the  contextual  and  content  matter  constraints  on  importance  were  identical  for  both 
versions.  These  constraints  reduce  the  potential  difference  in  the  importance  of  the  two 
versions  by  limiting  the  range  of  Importance  to  that  which  is  plausible  given  the  relationship 
of  the  sentence  content  to  the  passage  as  a  whole.  The  results  of  this  study  show  that  in 
addition  to  the  usual  contextual  and  subject  matter  constraints  which  influence  all  skilled 
readers,  novice  scientists  are  Influenced  in  a  systematic  and  reliable  manner  by  the  form  in 
which  *he  content  Is  presented  when  judging  importance  in  scientific  texts. 

These  findings  suggest  that  certain  variations  in  wording  which  do  not  affect  the 
reading  of  experts  or  domain-naive  readers  will  have  essentially  a  "signaling"  effect  on 
novice  attention  and  recall.  Signaling  involves  the  use  of  non-content  words,  such  as  "more 
importantly,"  'note  that,"  etc.,  to  emphasize  particular  text  information.  Signaling  has  been 
found  to  alter  readers’  attention,  as  indicated  by  differences  in  recall  patterns  for  signaled 
and  unslgnaled  texts  (Loman  &  Mayer,  1983,  Meyer,  1983).  The  results  of  the  current  study 
show  that  there  are  particular  alternative  wordings  (e.g.,  "is  defined  as"  vs.  "is  represented 
as")  which  do  not  function  as  signaling  phrases  for  most  readers  but  have  a  signaling  effect 
for  novice  readers  because  they  reflect  the  categorization  scheme  used  by  novices  to  assess 
Importance.  Therefore  authors  can  unintentionally  signal  particular  information  as  important 
through  wording  selections  of  this  type  if  they  are  unaware  of  novice  preconceptions 
regarding  the  text  content  domain. 

On  the  other  hand,  an  author  can  use  knowledge  of  novice  importance  rules  in 
conjunction  with  signaling  techniques  to  guide  readers'  attention  to  the  appropriate  text 
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content.  Writers  can  use  signaling  devices  to  emphasize  important  content  within  the 
sentence-type  categories  that  novices  consider  unimportant,  and  de-emphasize  less  critical 
Information  in  the  categories  that  novices  judge  as  Important.  Meyer  (1975)  describes  four 
types  of  signaling  techniques:  (1)  providing  cues  as  to  the  text  structure,  as  in  'the 
problem  is. ..the  solution  is,'  or  'first. ..second. ..third:'  (2)  paraphrasing  important  text  content 
before  it  is  presented,  such  as  "the  important  points  in  the  following  discussion  are...;'  (3) 
summary  statements  of  key  Ideas  presented  after  the  relevant  text;  and  (4)  pointer  words 
emphasizing  specific  statements  in  the  texts,  as  in  'more  Importantly,"  'it  is  notable  that," 
'unfortunately,'  etc.  These  devices  can  be  used  independently  of  the  overall  hierarchical 
structure  of  the  passage  to  emphasize  specific  content  within  each  level,  a  technique  that 
Meyer  (1983)  terms  a  'differential  emphasis  plan."  These  techniques  have  been  found  to 
be  effective  In  altering  recall  patterns  of  readers  of  varying  backgrounds  and  abilities  (Loman 
&  Mayer,  1983;  Meyer,  1983). 

Signaling  devices  can  be  used  by  writers  of  physics  texts  to  help  guide  novices' 
attention  to  the  important  content.  For  example,  the  current  study  indicates  that  definitions 
are  a  type  of  content  that  is  particularly  salient  to  novices.  Therefore,  it  would  be  helpful 
to  novices  if  writers  would  differentiate  the  less  important  definitions  from  those  that  are 
central  to  the  text.  This  could  be  done  by  including  structural  cues  or  preliminary 
summaries  that  emphasize  the  main  topics  or  content  that  the  author  wants  the  reader  to 
abstract  from  the  text,  and  thus  in  effect  "de-emphasize"  unimportant  definitions  This 
would  help  novices  distinguish  between  the  definitions  that  are  to  be  learned  (i.e.,  the  main 
points)  and  those  that  are  there  to  simply  aid  in  comprehension  or  elaborate  on  the 
important  points.  Facts,  on  the  other  hand,  are  not  particularly  salient  for  novices,  and 
earlier  research  indicates  that  novices  may  be  missing  important  content  of  this  type  (Dee- 
Lucas  4  Larkin,  1986).  In  this  case,  the  use  of  underlining  and  pointer  words  (e  g  .  note 
that,  It  is  Important  to  understand,  this  leads  to  the  important  conclusion  that,  etc.)  could  be 
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used  to  draw  attention  to  important  tacts. 

Another  technique  that  could  be  used  to  help  novices  distinguish  between  important 
and  unimportant  information  is  to  teach  novices  a  general  learning  schema  that  can  be 
applied  to  texts  to  assess  importance.  This  approach  to  manipulating  attention  and  learning 
is  "strategy-based"  rather  than  "text-based."  The  ability  of  students  to  acquire  general 
skills  for  learning  from  scientific  texts  has  been  demonstrated  by  Larkin  and  Reif  (1976). 
They  taught  students  a  skill  for  understanding  quantitative  relations  in  physics  texts  by  having 
students  work  through  a  series  of  training  materials  that  required  them  to  read  physics  texts 
and  answer  a  prespecified  set  of  questions.  The  subjects  were  able  to  learn  to  use  this 
question  set  as  a  strategy  for  acquiring  an  understanding  of  new  relations,  and  performed 
appreciably  better  on  tests  of  understanding  than  subjects  who  had  not  acquired  this 
learning  strategy.  In  the  case  of  the  current  study,  the  goal  of  the  learning  schema  would 
be  to  help  readers  distinguish  between  important  and  unimportant  definitions  and  tacts  This 
would  involve  teaching  students  what  types  of  definitions  ana  facts  should  be  learned  For 
example,  Larkin  and  Reif  were  able  to  specify  a  subset  of  facts  (i.e, ,  those  dealing  with 
units  and  typical  magnitudes)  that  are  particularly  important  for  understanding  quantitative 
relations.  This  type  of  schema  could  be  used  by  novices  in  reading  scientific  texts  to  aid 
in  identification  of  the  important  content  within  categories  of  information  (like  facts)  that  they 
generally  assume  are  unimportant.  This  schema  would  hold  true  for  a  broad  content  area 
within  the  domain  of  physics,  but  would  most  likely  vary  to  some  extent  with  subfields  of 
study  in  physics. 

The  present  study  indicates  that  novice  readers  can  be  sensitive  to  very  minor  changes 
in  wording  at  the  sentence  level  in  a  text,  in  that  these  changes  can  be  relevant  to 
distinctions  made  in  their  content  schema  for  the  text  domain  This  suggests  that  writers 
need  to  be  aware  of  the  manner  of  presentation  of  information  at  a  fairly  local  level  in  a 
text  In  particular,  experts  writing  for  a  novice  audience  will  be  most  effective  in  enhancing 
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learning  of  the  important  text  content  if  they  use  text-based  indicators  of  importance  in 
conjunction  with  knowledge  of  their  audiences'  content  schema.  In  this  way,  writers  can  use 
techniques  such  as  signaling  to  provide  novices  with  clear  signals  of  importance  which  will 
help  them  distinguish  between  important  and  less  critical  content. 
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Table  1:  Examples  of  definition  and  fact  versions  of  target  sentences. 

1.  Absolute  pressure  is  defined  as  simply  the  actual  pressure  at  a  point. 

Absolute  pressure  is  simply  the  actual  pressure  at  a  point. 

2.  Specific  gravity  is  defined  as  the  ratio  of  the  density  of  a  substance  to  the 
density  of  water. 

Specific  gravity  is  indicated  by  the  ratio  of  the  density  of  a  substance  to  the 
density  of  water. 

3.  In  terms  of  this  notation,  the  work  a  W  done  by  a  force  F  in  moving  an  object 
through  a  displacement  Ar  is  defined  as 

A  W  =  F*  Ar  =  F  (A  r  cos  9  ) 

In  terms  of  this  notation,  the  work  a  W  done  by  a  force  F  in  moving  an  object 
through  a  displacement  Ar  is  represented  as 

A  W  =  F-  Ar  =  F  (A  r  cos  9  ) 

4.  Specifically,  the  total  amount  of  work  done  in  changing  the  motion  of  a  particle 
is  defined  as  the  sum  of  the  works  done  by  each  of  the  individual  forces  acting 
on  the  particle. 

Specifically,  the  total  amount  of  work  done  in  changing  the  motion  of  a  panicle 

is  equal  to  the  sum  of  the  works  done  by  each  of  the  individual  forces  acting 

on  the  particle. 

5.  Intrinsic  quantities  are  defined  as  quantities  which  are  independent  of  volume. 
Intrinsic  quanitities  are  independent  of  volume. 

6.  The  unit  typically  used  for  measuring  work,  the  joule,  is  defined  as  the  work 

done  by  a  unit  force  (one  newton)  acting  on  a  unit  distance  (one  meter). 

The  unit  typically  used  for  measuring  work,  the  joule,  Indicates  the  amount  of 
work  done  by  a  unit  force  (one  newton)  acting  on  a  unit  distance  (one  meter). 

7.  Pressure  is  defined  as  the  magnitude  of  a  fluid  force  divided  by  the  area  of  the 
surface  on  which  it  acts. 

Pressure  can  be  calculated  by  dividing  the  magnitude  of  a  fluid  force  by  the 

area  of  the  surface  on  which  it  acts. 
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Table  2:  Parameter  estimates,  standard  errors,  and  ratio  of  estimates  to  standard 
errors  for  a  loglinear  model  of  the  ratings  data  from  the  experimental  groups. 
Asterisks  indicate  coefficients  that  differ  from  zero  by  more  than  2  standard  errors 

Ratio: 


Effect 

Coeff. 

St.  Error  Coeff./St. 

Error 

(a)Group 

novice  estimates  (expert  estimates  are  opposite) 

Rate  1 

.045 

.066 

.69 

Rate  2 

.074 

.075 

.98 

Rate  3 

.094 

.090 

1.04 

Rate  4 

.019 

.118 

.16 

Rate  5 

-.232 

.182 

-1.27 

(b)Type: 

definition 

estimates  (fact 

estimates  are  opposite) 

Rate  1 

.073 

.067 

1.10 

Rate  2 

.082 

.075 

1.10 

Rate  3 

-.224* 

.090 

-2.48 

Rate  4 

.064 

.115 

.56 

Rate  5 

005 

.192 

.03 

(c)Type  x  Group: 

definition  estimates  (fact  estimates  are  opposite)1 

Rate  1: 

Novice  (  +  ) 

Expert  (-) 

.133* 

.066 

2.01 

Rate  2: 

Novice  (•) 

Expert  (  +  ) 

.098 

.075 

1.30 

Rate  3: 

Novice  (  +  ) 

Expert  (-) 

.003 

.081 

.04 

Rate  4; 

Novice  (  +  ) 

Expert  (-) 

.014 

.117 

.12 

Rate  5: 

Novice  (-) 

.052 

.180 

.29 

Expert  (  +  ) 

'Parameter  estimates  indicate  the  size  of  the  effect;  positive  and  negative  symbols 
indicate  the  direction  of  the  effect  for  each  group. 
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Table  3:  Parameter  estimates,  standard  errors,  and  ratios  of  estimates  to  standard 
errors  for  a  loglinear  model  of  the  ratings  data  from  the  control  groups 
Asterisks  indicate  coefficients  that  differ  from  zero  by  more  than  2 

standard  errors. 


Ratio: 


Effect 

Coeff. 

St.  Error 

Coeff  ./St.  Error 

Group:  undergrad  estimates  (grad  estimates  are  opposite) 

Rate  1:  .323*  .065 

4.98 

Rate  2: 

.083 

.074 

1.12 

Rate  3: 

-.070 

.089 

-.79 

Rate  4: 

-.323* 

.117 

-2.76 

Rate  5: 

-.010 

.172 

-.06 

Level 

Level  1: 

Rate  1 

.695* 

.105 

6.62 

Rate  2 

.141 

.119 

1.18 

Rate  3 

-.302 

.156 

-1.94 

Rate  4 

-.492* 

.220 

-2.24 

Rate  5 

-.042 

.282 

-.15 

Level  2: 

Rate  1 

.128 

.105 

1.25 

Rate  2 

.013 

.118 

.11 

Rate  3 

.133 

.137 

97 

Rate  4 

-.047 

.188 

-.25 

Rate  5 

-.226 

281 

-.80 

Level  3: 
Rate  1 
Rate  2 
Rate  3 


.092 

.099 


e 


-.822* 

-.154 

169 


.119 


-8.92 
-1.56 
1  42 
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Table  4: 


Parameter  estimates,  standard  errors,  and  ratios  of  estimates  to  standard 
errors  for  a  logistic  regression  model  of  the  sentence  selection  data  from  t  e 

experimental  groups. 


Ratio 


Effect 

Coeff. 

St.  Error 

Coeff./St.  Error 

Type: 

-.414 

.099 

-4.18 

Group: 

-.186 

.099 

-1.88 

Type  x  Group: 

.236 

.099 

2.38 

Definition  (  +  ) 
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Table  5:  Parameter  estimates,  standard  errors,  and  ratios  of  estimates  to  standard 
errors  for  a  logistic  regression  model  of  the  sentence  selection  data  from  the 


control  groups 

Ratio: 

Effect 

Coeff. 

St.  Error 

Coett./St.  Error 

Type: 

-.220 

.098 

-2.24 

Type  x  Level: 

Level  1: 

-.448 

.139 

-3.22 

Level  2: 

.014 

.139 

.10 

Level  3: 

.462 

.140 

3.30 

Group: 

-.590 

.098 

-6.02 

Level: 

Level  1-2  dlff: 

.476 

.139 

3.42 

Level  2-3  dlff: 

1.316 

.140 

9.40 

\ ' 
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Figure  1:  Predicted  mean  ratings  for  the  type  x  group  interactions  in  the 

ratings  data. 
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Figure  2:  Predicted  proportion  of  target  sentences  selected  as  important  for 
the  type  x  group  interactions  In  the  sentence  selection  data. 


(b)  Control  Groups 
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Figure  3:  Predicted  proportion  of  target  sentences  selected  as  important  for 

the  type  x  level  interaction  in  the  sentence  selection  data  for  the  control 

groups. 
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